A Time Delay Neural Network model for simulating eye gaze data

Human eye movement modelling is a new, challenging, and promising research topic in computer vision. Human eye movement modelling aims at simulating the scan path in which a human being views an image, scene, or video. The successful modelling of human eye movements potentially benefits a wide range of applications such as image retrieval, image annotation, medical image diagnosis, and human visual perception. This paper presents a model based on a Time Delay Neural Network (TDNN) to simulate eye gaze data. Firstly, 120 Hz eye gaze data is acquired by a non-intrusive table mounted eye tracker. Our proposed model is to simulate a single subject’s image reading process. Seven features are then extracted based on knowledge of the human oculomotor system and the image contents to train a TDNN. Finally, the trained TDNN combined with a saccade control mechanism is used to simulate the scan path of a human being viewing an image. The proposed model can generate 600 points of raw eye gaze data in a five-second eye viewing window. Both subjective and objective methods are used to evaluate the model by comparing its behaviour and characteristics with the real eye gaze data collected from an eye tracker. Qualitative assessment shows the subjects can hardly tell the differences between the scan path from the model and that from a human being. By evaluating coincident probability and coincident significance , quantitative assessment shows the TDNN’s results are reasonable and similar to human scan paths.